Improving IV&V Techniques Through the Analysis of Project Anomalies: Bayes networks - preliminary report

The original plan of this work was the creation of a process by which conclusions learned from one IV&V project could be applied to another. In particular, we seek methods whereby an agent can say “that’s odd”; i.e. detect anomalies and propose repairs in active NASA projects. Given the current state of business knowledge and IV&V project data recorded at the IV&V facility, the methods proposed in the original plan (semantic web frame-based generalization and specialization over ontologies describing IV&V business practices) are not supportable, Hence, this report describes an alternate direction. Instead of working “top-down” from descriptions of business knowledge (which may never exist), or “bottom-up” from data (that may never be available), this project now focuses on “middle-out” and will try to combine the available data/models into a semantic whole. The currently available data/models are: – SILAP, from the IV&V planning and scoping team; – James Dabney’s Bayes networks that describe the IV&V business practices of the L3 IV&V contractor; – The PITS issue tracking data; – The LINKER database project that intends to join PITS to other data sources; – Balanced score card strategy maps from NASA Langley. – and the COCOMO data sets from JPL (notethese data sets are being explored elsewhere; hence, this project will not use its funds to directly explore COCOMO but may lever conclusions from that other project). This is a three year project that started in June 2006. After five months, and even after the required redirection described above, strong progress has been made. At SAS’06, a preliminary report described what had been learned from the SILAP data. This report presents a preliminary report on our use of the Dabney belief networks. It also offers some background notes on the entire problem. Subsequent reports will expand these preliminary reports into final conclusions. Those reports will focus on each of the above data sources, one by one, as well as exploring how to combine the above data sources into one anomaly detector. Acknowledgments: This work would have been impossible without James Dabney’s thorough modeling of IV&V processes in belied networks. Also, special thanks are owed to Wesley Deadrick for arranging access to the case study material. Credits: This research was conducted at West Virginia University under NASA sub-contract project 100005549, task 5e, award 1002193r. Cautions Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not constitute or imply its endorsement by the United States Government. Software:All software discussed here is available from http://unbox.org/wisp/qnet under the GNU Public License (version 2: see www.gnu.org/copyleft/gpl. html). Menzies: Bayes netspreliminary report 2 of 13